A distributed data store is generally a computer network in which data is stored on a plurality of data storage systems. Each data storage system is often referred to as a node in the distributed data store. Distributed data stores are highly scalable and less costly to maintain because the nodes can be easily added, removed, or replaced. As a result, distributed data stores are often used in data centers.
Although distributed data stores have been implemented using traditional Ethernet networks and traditional client-server architectures, such distributed data stores often suffered from high latency unless sufficient server CPU cores were provisioned to satisfy peak loads. This meant that server CPUs needed to be upgraded continually over time to satisfy increasing load demands, or the server clients needed to wait for additional server CPU resources to be spun up during load spikes.
Remote direct memory access (RDMA) offers a solution to the high latency problem of traditional Ethernet networks and traditional client-server architectures. RDMA allows direct memory access from the memory of one computer into that of another without involving either one's operating system. This permits high-throughput, low-latency networking, which is especially useful in large, parallel computer clusters. In view of these benefits, RDMA-enabled distributed data stores are increasingly being adopted.
In a traditional RDMA-enabled distributed data store, a data storage system generally has a single role during runtime. Specifically, each data storage system, or node, is typically pre-configured to operate either as a client or a server with respect to the rest of the data storage systems in the distributed data store. For example, each data storage system may be manually assigned either a client role or a server role before runtime. Then, during runtime, each data storage system operates according to its pre-assigned role. This means that a data storage system operating in server mode listens for incoming RDMA connections from a client, and a data storage system operating in client mode initiates RDMA connections with a server.